Ability-based Keyboards for Augmentative and Alternative Communication: Understanding How Individuals' Movement Patterns Translate to More Efficient Keyboards: Methods to Generate Keyboards Tailored to User-specific Motor Abilities.

This study presents the evaluation of ability-based methods extended to keyboard generation for alternative communication in people with dexterity impairments due to motor disabilities. Our approach characterizes user-specific cursor control abilities from a multidirectional point-select task to configure letters on a virtual keyboard based on estimated time, distance, and direction of movement. These methods were evaluated in three individuals with motor disabilities against a generically optimized keyboard and the ubiquitous QWERTY keyboard. We highlight key observations relating to the heterogeneity of the manifestation of motor disabilities, perceived importance of communication technology, and quantitative improvements in communication performance when characterizing an individual's movement abilities to design personalized AAC interfaces.

Prediction of Voice Fundamental Frequency and Intensity from Surface Electromyographic Signals of the Face and Neck.

Silent speech interfaces (SSIs) enable speech recognition and synthesis in the absence of an acoustic signal. Yet, the archetypal SSI fails to convey the expressive attributes of prosody such as pitch and loudness, leading to lexical ambiguities. The aim of this study was to determine the efficacy of using surface electromyography (sEMG) as an approach for predicting continuous acoustic estimates of prosody. Ten participants performed a series of vocal tasks including sustained vowels, phrases, and monologues while acoustic data was recorded simultaneously with sEMG activity from muscles of the face and neck. A battery of time-, frequency-, and cepstral-domain features extracted from the sEMG signals were used to train deep regression neural networks to predict fundamental frequency and intensity contours from the acoustic signals. We achieved an average accuracy of 0.01 ST and precision of 0.56 ST for the estimation of fundamental frequency, and an average accuracy of 0.21 dB SPL and precision of 3.25 dB SPL for the estimation of intensity. This work highlights the importance of using sEMG as an alternative means of detecting prosody and shows promise for improving SSIs in future development.

Ability-Based Methods for Personalized Keyboard Generation.

This study introduces an ability-based method for personalized keyboard generation, wherein an individual's own movement and human-computer interaction data are used to automatically compute a personalized virtual keyboard layout. Our approach integrates a multidirectional point-select task to characterize cursor control over time, distance, and direction. The characterization is automatically employed to develop a computationally efficient keyboard layout that prioritizes each user's movement abilities through capturing directional constraints and preferences. We evaluated our approach in a study involving 16 participants using inertial sensing and facial electromyography as an access method, resulting in significantly increased communication rates using the personalized keyboard (52.0 bits/min) when compared to a generically optimized keyboard (47.9 bits/min). Our results demonstrate the ability to effectively characterize an individual's movement abilities to design a personalized keyboard for improved communication. This work underscores the importance of integrating a user's motor abilities when designing virtual interfaces.

Surface Electromyography-Based Recognition, Synthesis, and Perception of Prosodic Subvocal Speech.

Purpose This study aimed to evaluate a novel communication system designed to translate surface electromyographic (sEMG) signals from articulatory muscles into speech using a personalized, digital voice. The system was evaluated for word recognition, prosodic classification, and listener perception of synthesized speech. Method sEMG signals were recorded from the face and neck as speakers with (n = 4) and without (n = 4) laryngectomy subvocally recited (silently mouthed) a speech corpus comprising 750 phrases (150 phrases with variable phrase-level stress). Corpus tokens were then translated into speech via personalized voice synthesis (n = 8 synthetic voices) and compared against phrases produced by each speaker when using their typical mode of communication (n = 4 natural voices, n = 4 electrolaryngeal [EL] voices). Naïve listeners (n = 12) evaluated synthetic, natural, and EL speech for acceptability and intelligibility in a visual sort-and-rate task, as well as phrasal stress discriminability via a classification mechanism. Results Recorded sEMG signals were processed to translate sEMG muscle activity into lexical content and categorize variations in phrase-level stress, achieving a mean accuracy of 96.3% (SD = 3.10%) and 91.2% (SD = 4.46%), respectively. Synthetic speech was significantly higher in acceptability and intelligibility than EL speech, also leading to greater phrasal stress classification accuracy, whereas natural speech was rated as the most acceptable and intelligible, with the greatest phrasal stress classification accuracy. Conclusion This proof-of-concept study establishes the feasibility of using subvocal sEMG-based alternative communication not only for lexical recognition but also for prosodic communication in healthy individuals, as well as those living with vocal impairments and residual articulatory function. Supplemental Material https://doi.org/10.23641/asha.14558481.

Motor unit drive: a neural interface for real-time upper limb prosthetic control.

OBJECTIVE: Modern prosthetic limbs have made strident gains in recent years, incorporating terminal electromechanical devices that are capable of mimicking the human hand. However, access to these advanced control capabilities has been prevented by fundamental limitations of amplitude-based myoelectric neural interfaces, which have remained virtually unchanged for over four decades. Consequently, nearly 23% of adults and 32% of children with major traumatic or congenital upper-limb loss abandon regular use of their myoelectric prosthesis. To address this healthcare need, we have developed a noninvasive neural interface technology that maps natural motor unit increments of neural control and force into biomechanically informed signals for improved prosthetic control. APPROACH: Our technology, referred to as motor unit drive (MU Drive), utilizes real-time machine learning algorithms for directly measuring motor unit firings from surface electromyographic signals recorded from residual muscles of an amputated or congenitally missing limb. The extracted firings are transformed into biomechanically informed signals based on the force generating properties of individual motor units to provide a control source that represents the intended movement. MAIN RESULTS: We evaluated the characteristics of the MU Drive control signals and compared them to conventional amplitude-based myoelectric signals in healthy subjects as well as subjects with congenital or traumatic trans-radial limb-loss. Our analysis established a vital proof-of-concept: MU Drive provides a more responsive real-time signal with improved smoothness and more faithful replication of intended limb movement that overcomes the trade-off between performance and latency inherent to amplitude-based myoelectric methods. SIGNIFICANCE: MU Drive is the first neural interface for prosthetic control that provides noninvasive real-time access to the natural motor control mechanisms of the human nervous system. This new neural interface holds promise for improving prosthetic function by achieving advanced control that better reflects the user intent. Beyond the immediate advantages in the field of prosthetics, MU Drive provides an innovative alternative for advancing the control of exoskeletons, assistive devices, and other robotic rehabilitation applications.

Development of sEMG sensors and algorithms for silent speech recognition.

OBJECTIVE: Speech is among the most natural forms of human communication, thereby offering an attractive modality for human-machine interaction through automatic speech recognition (ASR). However, the limitations of ASR-including degradation in the presence of ambient noise, limited privacy and poor accessibility for those with significant speech disorders-have motivated the need for alternative non-acoustic modalities of subvocal or silent speech recognition (SSR). APPROACH: We have developed a new system of face- and neck-worn sensors and signal processing algorithms that are capable of recognizing silently mouthed words and phrases entirely from the surface electromyographic (sEMG) signals recorded from muscles of the face and neck that are involved in the production of speech. The algorithms were strategically developed by evolving speech recognition models: first for recognizing isolated words by extracting speech-related features from sEMG signals, then for recognizing sequences of words from patterns of sEMG signals using grammar models, and finally for recognizing a vocabulary of previously untrained words using phoneme-based models. The final recognition algorithms were integrated with specially designed multi-point, miniaturized sensors that can be arranged in flexible geometries to record high-fidelity sEMG signal measurements from small articulator muscles of the face and neck. MAIN RESULTS: We tested the system of sensors and algorithms during a series of subvocal speech experiments involving more than 1200 phrases generated from a 2200-word vocabulary and achieved an 8.9%-word error rate (91.1% recognition rate), far surpassing previous attempts in the field. SIGNIFICANCE: These results demonstrate the viability of our system as an alternative modality of communication for a multitude of applications including: persons with speech impairments following a laryngectomy; military personnel requiring hands-free covert communication; or the consumer in need of privacy while speaking on a mobile phone in public.

Contribution from motor unit firing adaptations and muscle coactivation during fatigue.

The control of motor unit firing behavior during fatigue is still debated in the literature. Most studies agree that the central nervous system increases the excitation to the motoneuron pool to compensate for decreased force contributions of individual motor units and sustain muscle force output during fatigue. However, some studies claim that motor units may decrease their firing rates despite increased excitation, contradicting the direct relationship between firing rates and excitation that governs the voluntary control of motor units. To investigate whether the control of motor units in fact changes with fatigue, we measured motor unit firing behavior during repeated contractions of the first dorsal interosseous (FDI) muscle while concurrently monitoring the activation of surrounding muscles, including the flexor carpi radialis, extensor carpi radialis, and pronator teres. Across all subjects, we observed an overall increase in FDI activation and motor unit firing rates by the end of the fatigue task. However, in some subjects we observed increases in FDI activation and motor unit firing rates only during the initial phase of the fatigue task, followed by subsequent decreases during the late phase of the fatigue task while the coactivation of surrounding muscles increased. These findings indicate that the strategy for sustaining force output may occasionally change, leading to increases in the relative activation of surrounding muscles while the excitation to the fatiguing muscle decreases. Importantly, irrespective of changes in the strategy for sustaining force output, the control properties regulating motor unit firing behavior remain unchanged during fatigue. NEW & NOTEWORTHY This work addresses sources of debate surrounding the manner in which motor unit firing behavior is controlled during fatigue. We found that decreases in the motor unit firing rates of the fatiguing muscle may occasionally be observed when the contribution of coactive muscles increases. Despite changes in the strategy employed to sustain the force output, the underlying control properties regulating motor unit firing behavior remain unchanged during muscle fatigue.

Silent Speech Recognition as an Alternative Communication Device for Persons with Laryngectomy.

Each year thousands of individuals require surgical removal of their larynx (voice box) due to trauma or disease, and thereby require an alternative voice source or assistive device to verbally communicate. Although natural voice is lost after laryngectomy, most muscles controlling speech articulation remain intact. Surface electromyographic (sEMG) activity of speech musculature can be recorded from the neck and face, and used for automatic speech recognition to provide speech-to-text or synthesized speech as an alternative means of communication. This is true even when speech is mouthed or spoken in a silent (subvocal) manner, making it an appropriate communication platform after laryngectomy. In this study, 8 individuals at least 6 months after total laryngectomy were recorded using 8 sEMG sensors on their face (4) and neck (4) while reading phrases constructed from a 2,500-word vocabulary. A unique set of phrases were used for training phoneme-based recognition models for each of the 39 commonly used phonemes in English, and the remaining phrases were used for testing word recognition of the models based on phoneme identification from running speech. Word error rates were on average 10.3% for the full 8-sensor set (averaging 9.5% for the top 4 participants), and 13.6% when reducing the sensor set to 4 locations per individual (n=7). This study provides a compelling proof-of-concept for sEMG-based alaryngeal speech recognition, with the strong potential to further improve recognition performance.

A combined sEMG and accelerometer system for monitoring functional activity in stroke.

Remote monitoring of physical activity using body-worn sensors provides an alternative to assessment of functional independence by subjective, paper-based questionnaires. This study investigated the classification accuracy of a combined surface electromyographic (sEMG) and accelerometer (ACC) sensor system for monitoring activities of daily living in patients with stroke. sEMG and ACC data (eight channels each) were recorded from 10 hemiparetic patients while they carried out a sequence of 11 activities of daily living (identification tasks), and 10 activities used to evaluate misclassification errors (nonidentification tasks). The sEMG and ACC sensor data were analyzed using a multilayered neural network and an adaptive neuro-fuzzy inference system to identify the minimal sensor configuration needed to accurately classify the identification tasks, with a minimal number of misclassifications from the nonidentification tasks. The results demonstrated that the highest sensitivity and specificity for the identification tasks was achieved using a subset of four ACC sensors and adjacent sEMG sensors located on both upper arms, one forearm, and one thigh, respectively. This configuration resulted in a mean sensitivity of 95.0%, and a mean specificity of 99.7% for the identification tasks, and a mean misclassification error of < 10% for the nonidentification tasks. The findings support the feasibility of a hybrid sEMG and ACC wearable sensor system for automatic recognition of motor tasks used to assess functional independence in patients with stroke.

The electromyographic signal as a presymptomatic indicator of organophosphates in the body.

Organophosphate (OP) compounds are present in household and agricultural pesticides as well as in nerve agents. The toxic effects of these chemicals result from their anticholinesterase activity, which disrupts nerve junctions and parasympathetic effector sites, leading to a variety of symptoms and possible death. When the anticholinesterase agents in OP compounds reach the neuromuscular junction, they cause a disruption in the firing of muscle fiber action potentials. This effect has the potential of altering the time course of the electromyographic (EMG) signal detected by surface electrodes. We investigated the association between OP compound dose, surface EMG changes, and overt signs of OP toxicity. Daily doses of 10-15 microg/kg of diisopropylfluorophosphate (DFP) were injected into the calf muscle of four rhesus monkeys while surface EMG signals were recorded from two thigh muscles bilaterally. With increasing number of doses, the EMG signal presented an increasing number of time gaps. The presence of the gaps was evident prior to any overt symptoms of cholinesterase toxicity. These findings can lead to the development of noninvasive technology for indicating the presence of OP compounds in muscle tissue prior to clinical abnormalities.